Low-Rank Approximations with Sparse Factors I: Basic Algorithms and Error Analysis

We consider the problem of computing low-rank approximations of matrices. The novel aspects of our approach are that we require the low-rank approximations be written in a factorized form with sparse factors and the degree of sparsity of the factors can be traded oo for reduced reconstruction error by certain user determined parameters. We give a detailed error analysis of our proposed algorithms and compare the computed sparse low-rank approximations with those obtained from singular value decomposition. We present numerical examples arising from some application areas to illustrate the eeciency and accuracy of our algorithms. 1. Introduction. We consider the problem of computing low-rank approximations of a given matrix A 2 R mn which arises in many applications areas; see 5, 14, 17] for a few examples. The theory of singular value decomposition (SVD) provides the following characterization of the best low-rank approximations of A in terms of Frobenius norm k k F 5, Theorem 2.5.3]. Theorem 1.1. Let the singular value decomposition of A 2 R mn be A = UV T , i=k+1 2 i = minf kA ? Bk 2 F j rank(B) kg: The minimum is achieved with best k (A) U k diag(1 ; : : : ; k)V T k ; where U k and V k are the matrices formed by the rst k columns of U and V , respectively. For any low-rank approximation B of A, we call kA ? Bk F the reconstruction error of using B as an approximation of A. By Theorem 1.1, best k (A) has the smallest reconstruction error in Frobenius norm among all the rank-k approximations of A. In certain applications, it is desirable to impose further constraints on the low-rank approximation B in addition to requiring that it be of low-rank. Consider the case where, for example, the matrix A is sparse; it is generally not true that